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APPARATUS FOR VIDEO ACCESS AND CONTROL OVER 
COMPUTER NETWORK, INCLUDING IMAGE CORRECTION 

CROSS-REFERENCE TO RELATED APPLICATIONS 

This application claims priority based on U.S. Provisional Patent Application 
Serial No. 60/025,604, filed September 9, 1996, entitled "Apparatus For Video Access 
And Control Over Computer Network", and this application claims priority based 
on U.S. Provisional Application Serial No. 60/033,485, filed December 20, 1996, 
entitled "Apparatus For Video Access And Control Over Computer Network, 
Including Image Correction". Both provisional applications are incorporated by 
reference in their entirety. 

INCORPORATION BY REFERENCE 

Additionally, the following patents, patent applications and publications are 
incorporated herein by reference: 

U.S. Patent No. 5,559,549, issued September 24, 1996 to Hendricks et al., 
U.S. Patent 5,600,573, issued on February 4, 1997 to Hendricks et al., 
U.S. pending patent application Serial No. 08/352,205, filed 

December 2, 1994, entitled NETWORK MANAGER FOR CABLE 

TELEVISION SYSTEM HE ADENDS, now , 

U.S. Patent No. 5,185,667, issued February 9, 1993 to Zimmerman, 
U.S. Patent No. 5,313,306, issued May 17, 1994 to Kuban et al, 
U.S. Patent No. 5,359,363, issued October 25, 1994 to Kuban et al., 
U.S. Patent No. 5,384,588, issued January 24, 1995 to Martin et al., 
U.S. Patent No. 5,489,940, issued February 6, 1996 to Richardson et al., 
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PCT Publication No. WO 96/07269, published March 7, 1996 by Jambhekar et aL, 
PCT Publication No. WO 96/08105, published March 14, 1996 by Labun, 
PCT Publication No. WO 96/18262, published June 13, 1996 by Richardson et al, 
PCT Publication No. WO 96/21173, published July 11, 1996 by Harris et aL, and 
PCT Publication No. WO 96/21205, published July 11, 1996 by Harris et al.. 

BACKGROUND OF THE INVENTION 

This invention relates to the distribution of audiovisual signals through 
communications networks such as computer networks and servers. The invention 
has particular use with respect to global networks such as the internet and "World 
Wide Web". The invention also relates to education. Particularly, the invention 
provides an alternative to in-person classroom instruction. 

1. Field Of The Invention 

The present invention relates to the fields of education, audiovisual systems, 
communications systems and computer networks. 

Individuals from around the world exchange ideas and information with each 
other in order to learn more about other people, cultures, and the environment in 
which we live. Video and audio signals are commonly transmitted over broadcast 
communications media to provide viewers with news and entertainment. 
Computer networks are used for the remote exchange of data and other information. 
Broadly speaking, these systems are attempts to communicate useful knowledge 
between geographically separate individuals and institutions. The invention 
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generally relates to improvements in the transmission of information between 
remote locations. 



2. Description Of Related Art 

There is a constant desire to improve education and knowledge at all levels. 
It is thought that true human progress can only be achieved if people's 
understanding of each other is improved and if people's understanding of nature 
and the environment is improved. Traditionally, education and knowledge have 
been obtained in schools from classroom instruction and from the reading of boqks. 

The disadvantage of current classroom instructional systems is that students 
must be physically present in the classroom to participate in the educational 
process. Therefore, students who are geographically displaced from the location of 
the classroom often cannot attend class instruction as often or as timely as students 
who are nearby to the classroom. 

The disadvantage of textbooks is that they are often not kept current with 
recent events or technological changes. Textbooks are usually only updated on a 
yearly or less frequent basis, while important changes may occur monthly or more 
frequently. Also, to save funds, schools may not purchase new textbooks even 
though the textbooks have been updated. Therefore, the new knowledge, although 
available, is not communicated to students. 

Recently, audiovisual presentations have begun to be used in the field of 
education. These systems may provide playback of a recording of a lecturer who 
provides a presentation on an educational topic. For example, students may learn 
about math from watching a videotape or television broadcast of a math professor's 
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lecture. Education can also occur on a more informal basis. For example, specialty 
channels in the United States such as the Discovery Channel® and The Learning 
Channel® (headquartered in Bethesda, Maryland, U.S.A.) broadcast educational 
programming which both entertains and educates a diverse viewership. 

The disadvantage of these audiovisual systems is that they are not interactive. 
Students are unable to ask questions, and the lecturer is unable to tailor the 
presentation of material to the specific needs of the current student audience. 
Consequently, the needs of the students are not met. 

Cable and broadcast television are commonly known media which supply 
information to large numbers of viewers equipped with receivers known as 
"television sets." By receiving a broadcast, cablecast or satellite signal, users are 
able to view scenes from remote locations and observe newsworthy events which 
occur far from the user's location. However, conventional television is a one-way 
media in which users cannot communicate with each other or the broadcaster. 

Recently, the advent of the "internet," and "World Wide Web," in conjunction 
with the proliferation of personal computers, has allowed people to exchange 
information and ideas on a global and inexpensive basis. Generally speaking, the 
internet is a large computer network which connects "host" computers. Users with a 
computer, modem and telephone line commonly call via telephone to connect with a 
"host." The "host," being in communication with other hosts (connected to other 
users) is able to transfer information between users. The internet is used, for 
example, to transfer, data files, still images, sounds and messages between virtually 
any two points in the world with telephone access. 
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The use of the internet has increased dramatically since 1981, when 
approximately 300 host computers were linked together. It has been estimated that 
in 1989, the number of linked host computers was fewer than 90,000; but by 1993, 
over a million host computers were connected. Currently over 9.4 million host 
computers are linked (not including the personal computers people use to access 
these hosts via modems) and as many as 40 million people around the world may 
have access to the internet medium. This number is expected to grow to 200 million 
by the year 1999. 

Users on the internet are able to transfer text, graphics, and still pictures 
between remote locations. Other types of information which can be transmitted 
include files containing prerecorded sequences of images. To view these images, 
users download a large data file, and after running appropriate software, see a 
sequence of images on the computer screen. These images are not provided in real 
time, and are not viewable while the user is accessing the internet. 

Therefore, even though the internet is a two-way communication medium, it 
is not currently being utilized to provide video information and audiovisual 
presentations. This is a disadvantage, in that a large number of people have been 
accustomed to television audiovisual presentations, and prefer an audio-video 
presentation to a textual or graphical presentation. 

What is needed is a medium of communication that is interactive and which 
carries audio, video, text, and graphics. 

What is needed is an educational system which is user friendly and 
entertaining. 
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What is needed is to improve the internet such that users can access many 
audiovisual programs. 

What is needed is to provide users with live video from remote sites. 

What is needed is a remote video system with increased realism and accuracy, 
such that users feel as though they were actually present at the remote location. 

SUMMARY OF THE INVENTION 

In accordance with the present invention, video is collected at a remote 
site. (The term "video", as used herein, includes stereophonic or monophonic audio 
signals which may accompany a video signal. Additionally, "video" is used broadly 
herein to include still images, groups of related still images, animation, graphics, 
pictures, or other visual data.) The remote video information may be obtained from 
a video cassette, CD ROMs, television channels, one or more video cameras, or other 
well known sources. If video cameras are used, they may be connected to a 
computer so that they are remotely controllable, or they may be oriented such that a 
perception of control can be created for users. The video may relate to remote sites 
of interest, such as a pyramid in Egypt, or the images may relate to an educational 
lecture being conducted at a remote site. 

The collected video is transferred to a web site, either in compressed or 
uncompressed form. The video may be physically transported or may be 
transmitted through a communications medium to the web site. 

The web site contains a storage media which may store some or all of the 
video. Additionally, the web site passes camera control commands, if applicable, to 
the remotely controlled cameras or may simulate the remote control of a camera. 



WO 99/12349 PCT/US98/18271 

The main function of the web site is to pass video to a plurality of users, through a 
communication media such as the internet, in response to user selections. The video 
passed to the plurality of users may be live video being fed to the web site, or may 
be stored video. A number of video servers are used to output the video to the users 
through the communications media, such as the internet. The video may be tailored 
by the web site for the particular user's hardware, including data communication 
equipment, or memory size, etc., i.e. the data rate matches the highest speed which 
the user's equipment can handle. 

Users receive and display the video sent from the web site. Many 
simultaneous video pictures may be received. Of course, the quality and frame rate 
of the video is dependent on the user's communications hardware. Users with high- 
speed modems or cable modems receive higher quality video. The users are able to 
send commands and /or queries to the web site. The commands and queries are 
forwarded to remote locations to control remote cameras or query remotely located 
instructors. Alternatively, the commands cause the web site to change from among 
many video signals with different camera angles or locations (or to transmit a 
different portion of a wide angle image), causing the user to have a perception of 
remote camera control The user's commands may also cause a different portion of a 
received wide angle image to be displayed, giving the user a perception of camera 
control. 

In addition to video, the web site provides information, such as graphics and 
text, which is related to the video. This information may be automatically supplied, 
or provided upon user request. Therefore, the user is provided with a 
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comprehensive set of information concerning remote sites, enabling the user to be 
quickly educated about the remote site of interest. 



BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram of an embodiment of the invention where remote 
video is provided to a web server by videocassette and by ordinary television. 

Figure 2 is a block diagram of an embodiment of the invention where remote 
video is provided by remotely located cameras and a communication network 
carries the video to the web server. 

Figures 3A and 3B are a block diagrams of an embodiment of the invention 
using the embodiments of Figures 1 and 2 with remotely controllable cameras. 

Figure 4 shows remote cameras positioned around a building for perceived 
camera control. 

Figures 5A, 5B, 5C, and 5D show video images from specific cameras shown 
in Figure 4. 

Figure 6 shows remote cameras deployed to follow a parade route. 

Figures 7 A and 7B show remotely controlled cameras at a remote location. 

Figures 8 A and 8B show a single remote camera at a remote location, where 
the camera has a 180 degree spherical (or other wide angle) lens. 

Figures 9 A and 9B are block diagrams of a server platform. 

Figure 10 is a block diagram of communications paths from the server site to 
remote users. 
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Figure 11 shows a home page in accordance with an embodiment of the 
invention. 

Figure 12 shows a "society" page in accordance with another embodiment of 
the invention. 

Figure 13 shows a "map" page of remote camera locations throughout the 

world. 

Figure 14 shows a "watch" page containing live video feeds from five remote 
cameras. 

Figure 15 shows a page directed to determining the user's data rate. 
Figure 16 shows a page of an interactive lecture. 

Figures 17 and 18 show pages of an embodiment of the invention which 
combines live video, prestored video, graphics, and interactive questions. 

Figure 19 shows a flow diagram of a method of automatically monitoring and 
panning an area using perceived camera control. 

Figure 20 is an exemplary screen display of the present invention, showing 
video and also showing video data. 

Figure 21 is a diagram showing the interaction between a computer network 
embodiment of the present invention and a cable television system. 

DETAILED DESCRIPTION OF THE DRAWINGS 

As stated previously, the present invention is related to obtaining video from 
remote sites and interactively presenting that video to users. The video is obtained 
at a remote site, communicated to a web site (where it may be stored), and 
forwarded to users. 
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1. Obtaining Video From Remote Sites, Communicating the Video to a 
Web Site, and Streaming the Video To Users, 

Figure 1 shows a preferred embodiment of the invention where remote video 

sources are videocassette and television programs. Figure 1 shows remote sites 102, 

remote cameras 104, videocassette 106, compression devices 108, 114, digital storage 

device 110 and web site 112. As shown in Figure 1, a video camera 104 is used to 

film activity at remote site 102. As discussed below, numerous video cameras at a 

single remote site may be used to obtain different views and audio (preferably 

stereophonic) of the remote site from different angles and orientations. Also, 

numerous remote sites, each with its own video camera, may used as shown at 102', 

102" and 104' and 104". The video cameras film events at the remote sites, and 

record the events on videocassette 106 or other suitable media. 

The recorded information is then transported to a web site 112, or to a site in 
communication with web site 112. As shown in Figure 1, the recorded information 
from video tape 106 is then compressed in compression unit 108 and stored in digital 
storage media 110. Many compression algorithms may be used, such as MPEG-1, 
MPEG-2 and Wavelet. Compression systems currently available from The Duck 
Corp, Xing Technology Corp., Indeo, Digital Video Arts, Ltd., VDOnet Corp. and 
Intel Corp., may be used with the system. The digital storage media may be any 
known storage device, such as a hard disk, CD ROM, digital video disc (DVD), 
digital tape, video file server or other media. 

The stored and compressed audio/video is then provided on a number of 
streamed audio-video outputs 116 from the web site 112. This enables many users to 
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access the stored video and audio, and allows for one user to receive numerous 
audio-video signals, i.e. split the display into numerous "camera" feeds. 

In addition to providing streamed audio and video from videocassette, the 
web site 112 may provide audio and video from television channels. The television 
signals are received by a conventional television receiver (not shown), and digitally 
compressed by the compression unit 114 and fed through the web site 112 to the 
streamed output. It is not normally necessary to store the television programs in a 
digital storage unit (such as the storage unit 110), since the audio and video is 
constantly incoming and changing. However, certain segments of broadcast 
television may be stored in a storage device (not shown) for recall by a user. 

Figure 2 shows another embodiment of the invention where similar reference 
numerals indicate items that correspond to the items shown in Figure 1. The system 
of Figure 2 uses remote cameras and a communication network to provide remote 
video to the web site. Figure 2 shows remote sites 102, video cameras 104, 
compression unit 118, data communication network 120, web site 130, digital storage 
unit 132, and streamed video 116. 

As shown in Figure 2, remote sites 102 are filmed by cameras 104 (as in Figure 
1). However, in this embodiment, the output of the cameras 104 pass through a 
compression unit 118. The compressed audio and video is communicated over data 
communication network 120 to web site 130. The data communication network 120 
may be any network currently known to one of ordinary skill in the art, such as 
land-leased lines, satellite, fiber optic cable, microwave link or any other suitable 
network. 
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Other suitable networks may be cellular networks or paging networks. In a 
paging network, cameras 104 may be connected to a paging device and/or digital 
storage media or paging transmitter for communication of the video (including still 
images) to the web site 130. The following publications, hereby incorporated by 
reference, disclose relevant systems: PCT Publication No. WO 96/07269, published 
March 7, 1996 by Jambhekar et al; PCT Publication No. WO 96/21173, published 
July 11, 1996 by Harris et al.; PCT Publication No. WO 96/21205, published July 11, 
1996 by Harris et al. 

The web site 130 in this example is adapted to receive information from the 
data communication network 120. The web site may pass the video from cameras 
104 to users at streamed video outputs 116. In alternative embodiments, the web site 
may contain a decompressor to decompress the video prior to streaming it to users, 
or change the compression scheme of the video to one which is compatible with the 
connected user. Alternatively, the video may be compressed at the streamed video 
output and users who connect to the web site 130 may run decompression software. 
The web site 130 may store the audio and video received over data communication 
network 120 in digital storage unit 132 before providing it to the streamed outputs 
116. Alternatively, the audio and video may be directly passed to the streamed 
outputs 116. 

Figure 3A shows another embodiment of the invention that combines the 
embodiments of Figures 1 and 2 and adds remote camera control. Figure 3A shows 
remote sites 102, cameras 104, computer 134, video path 122, 129, control path 124, 
126, 128, compressors 108, 114, 118, 136 data communication network 120, web site 
140, digital storage means 132, and streamed video 116. As with Figures 1 and 2, 
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remote sites 102 are filmed by camera 104. As with Figure 1, the web site 140 is able 
to receive video tape 106, compress the audio and video in compression unit 108, 
and store the compressed audio and video 110. Audio and video from television 
stations may also be compressed by compression unit 114 and stored or passed as 
streamed video 116, as in Figure 1. 

Likewise, the cameras 104 may be connected to compression unit 118 (as in 
Figure 2) and communicate compressed audio and video to web site 140 via data 
communication network 120. Thus the functions performed by the embodiments 
shown in Figures 1 and 2 may be combined in a variety of manners at a single web 
site 140. 

Figures 3A and 3B add the additional feature of camera control to the 
previously described embodiments. As shown in Figure 3A, a computer 134 is 
connected to remote camera 104. The computer is able to control a mechanical or 
electrical device on the camera 104, to alter the camera's orientation (including 
position and/or angle). Audio and video from the camera 104 passes to the 
computer 134. The video may be processed and stored in the computer. Preferably, 
as shown in Figure 3B, the computer is connected to multiple remote cameras 104' 
and 104" so that multiple users may each control a camera. The computer 134 may 
either contain a compressor or be connected to an external compression unit 136. 
The video from cameras 104' and 104" is compressed and provided to data 
communications network 120. This compressed video is subsequently received by 
web site 140. The remote cameras 104', 104" (Figure 3B) may be controlled by 
control signals passed from computer 134 on path 124. The control signals are 
received by computer 134 from the data communications network 120 over the 
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camera control path 126. The web site 140 provides the control information to the 
data communications network 120 over path 128. The web site 140 of this example is 
adapted to pass control signals to cameras 104 and to store video images in a digital 
storage means 132. The web site provides a number of streamed video outputs 116 
as in the other examples. 

This embodiment allows remote users to control the angle or orientation of 
cameras 104', 104". Users are connected to the web site 140 and receive the streamed 
video 116 from the cameras 104', 104". If the users wish to move the camera 104', 
104" to the right, they may enter a user command (such as "pan right") at their 
terminal. The command is received by the web site 140, and formatted, if necessary. 
The command is outputted to the data communication network 120 as a control 
signal through the camera control path 128. The remote computer 134 receives the 
camera control signals from the communication network 120 over camera control 
path 126. The remote computer 134 may be adapted to control multiple cameras at 
multiple locations 102, or multiple cameras at the same location 102. 

The computer 134 is connected to the remote camera 104 by a camera control 
path 124. This path allows control commands from the computer to travel to the 
cameras 104', 104" and control the cameras 104', 104". The cameras 104', 104" may 
have computer-controlled swivel motors (not shown) for panning left and right, may 
have a computer-controlled pivot motor (not shown) for panning up and down, and 
may have a computer-controlled motor (not shown) for moving a zoom lens. These 
motors are known to the artisan and are currently available. A plurality of cameras 
may be provided at a single site to allow multiple users to have camera control at the 
same time. 
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t. 

This system of obtaining and/or storing video at a web site is extremely 
flexible. The system allows for perceived camera control by multiple cameras, actual 
camera control of one or more cameras, perceived camera control via a wide-angle 
lens on a single camera, and for the generation of comprehensive interactive 
programs. 

2. Perceived Camera Control With Multiple Cameras. 

In one alternative embodiment, shown more clearly in Figures 4-6, users are 
given the perception of camera control. To achieve this, a plurality of fixed cameras 
104, 150, 152, 153, 154, 156, 158, 160, 162 (Figure 4) are disposed around a remote site 
102. In accordance with this embodiment, it appears to users that they are 
controlling the angle or position of a camera when in actuality they are merely being 
transferred to the video output of a different camera. Figures 4-6 show this concept 
in greater detail. 

As shown in Figure 4, a building 146 is being prepared for demolition. 
Disposed around the building 146 are cameras 104, 150, 152, 153, 154, 156, 158, 160, 
162, connected to a computer 135. The computer 135 is connected to a 
communication network 120 (not shown). The video from cameras 104, 150, 152, 
153, 154, 156, 158, 160, 162 is digitized and preferably compressed prior to 
communication over network 120, either by compressors connected to the cameras 
(not shown) or by a compressor connected to the computer 135 (not shown). The 
cameras may be digital cameras or analog cameras connected to an analog-to-digital 
converter. 
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The cameras specifically identified around the periphery are cameras 150, 152, 
153, 154, 156, 158, 160, and 162. For reference, the building contains the letter "A" 
and the letter "B" on two sides as shown at 144 and 148 in Figures 4 and 5. A number 
of additional cameras 104 are disposed about the periphery of the building in a 
circular pattern. The pattern and number of cameras are not critical, but will control 
how the user perceives movement of the "camera". 

Referring to Figure 4, a video camera 150 faces side A, a video camera 152 is 
between sides A and B, a video camera 153 faces side B and a video camera 154 is 
between side B and the side opposite side A. The video cameras 156, 158, 160 and 
162 are disposed closer to the building, as shown. All the video cameras contain 
audio pickups (preferably stereo). Additionally, all the video cameras are connected 
to a computer 135 which outputs compressed audiovisual signals to the 
communication network 120 and consequently to the web site. The system shown in 
Figure 4 may be implemented by the systems shown in either Figure 2 or Figure 3. 
Any number of users in communication with the web site 130, 140 may receive the 
audio and video from these cameras. 

Figure 5A shows a typical screen view 150 of the video presented to remote 
users who are connected to the web site of the present invention. As shown, the user 
is observing live video from camera 150, which provides a view of the building on 
side A. A "toolbar" of commands 151 is presented to the user, including a pan left 
command a pan right command "□", a pan up command "□" and a pan down 
command "□". An "autopan" command is used in conjunction with another 
command (such as pan right). The "autopan" command is used to automatically 
move the picture position in the direction previously entered. For example, if 
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"autopan" is entered after "pan right," then the picture will keep panning right until 
another key is pressed or a default key (such as the ESCape key) is pressed. The 
speed of the "autopan" function is controlled by the "speed" command, which is used 
in conjunction with the "+" and "-" commands. Additionally, the "+" and "-" 
commands, when used alone, control a "zoom-in" and "zoom-out" function, 
respectively. The "toolbar" commands are selected via a user input device, which 
may be a keyboard, mouse, trackball, remote control, etc... 

When any user wishes to switch from the view of the camera 150 (Figure 5A) 
and pan to the right, the user initiates a pan right command "□", which is 
transmitted to the web site 130, 140 (Figures 2 and 3). The web site receives the 
command, and in response, causes the video from the camera positioned to the right 
of the camera 150, in this case the video camera 152 (Figure 4) to be transmitted to 
the user. The user then observes the picture appearing in Figure 5B, which appears 
to be a view to the right from the previous position (camera 150). If the user 
continues to pan right, he is presented with the Figure 5C view, received from the 
camera 153. The user may continue to pan right all away around the building in this 
manner. 

Additionally the user has special functions available, such as "autopan" and 
"zoom." For example, "autopan" in conjunction with "pan right" would cause the 
view of the building to rotate, at a speed dictated by the "speed" function and the 
"+" and "-" keys. Using the "+" and "-" keys alone causes the view to change to a 
closer camera ("+") or a camera further away ("-"). As shown in Figure 4, the 
cameras 156, 158, 160 and 162 are disposed closer to the building than cameras 150, 
152, 153 and 154. A "magnified" image, obtained from the camera 156, is shown in 
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Figure 5D. If no cameras are disposed closer or further away, digital image 
processing may be used to digitally increase or reduce the size of the image- The 
software which controls these functions may be disposed either at the web server or 
on the user's computer. 

Thus, users may obtain different views of the building 146 as if they were 
remotely controlling the positioning of a single remote camera. The users may 
observe the demolition of the building from many exciting perspectives. This 
"perceived" camera control is advantageous because it allows any number of users to 
"control" a camera. A single camera which is remotely controllable is only 
controllable by a single user. Thus, the present invention is suitable for large 
audiences. The realism of this perceived control is directly dependent upon the 
number of cameras and their distances from the viewed object. 

Therefore, when the building 146 is demolished, any number of users may 
pan around the building in real time as if they were actually present at the site. 
When the building is demolished, the video cameras pick up, preferably in stereo, 
the sounds of the demolition. Users who have loudspeakers connected to their 
computer may experience the demolition almost as if they were present. 

Figure 6 shows a deployment of a number of cameras 104 which are arranged 
in a linear fashion around a point of interest, each camera connected to computer 135 
as in Figure 4. As with Figures 4-5, this embodiment uses "perceived" camera 
control which may be achieved by the systems shown in Figures 2 or 3. In this 
example, the remote location and point of interest is a parade, such as a New Year's 
Day Parade. With the camera deployment shown, a user may traverse the length of 
the parade without actually being present. Users may view whichever part of the 
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parade they are interested in, for as long as they desire, without worry that they 
have missed an interesting band or float. In this example, the camera deployment 
merely follows the parade route. Parents who have children in a band or float may 
search for the child and follow the child throughout the parade route, rather than 
having to monitor every moment of the parade on television in the hopes that the 
child will pass the reviewing camera when the parents are watching. The parents 
merely "move" from different cameras along the parade route as their children 
progress in the parade. 

3. Actual Camera Control of Single /Multiple Cameras. 

Figures 7 A and 7B show another embodiment, where a number of cameras 
160, 162, 164, 166, are provided. These cameras are in direct communication with 
and are controlled by computer 170. Although it is possible to form a ring of 
cameras to perform "perceived" camera control (as in Figures 4-6), the embodiment 
shown uses four cameras 160, 162, 164, 166 which contain motors 105 (Figure 7B) for 
controlling the camera's positioning. The motors are controlled by computer 170. 
Either a single computer 170 or a number of computers 170 may be used. The 
remote location and point of interest shown in Figures 7A and 7B are, for example, a 
watering hole or desert oasis. Users who access the web site 140 are able to observe 
live video of wildlife behavior at the watering hole. The cameras 160, 162, 164, 166 
are disposed at an island in the middle of the watering hole. The toolbar 151 of 
Figure 5 is also used in this embodiment and enables users to choose camera control 
commands to spin the cameras around or perform other camera functions, such as 
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zoom. Users are therefore able to receive different views and angles, and observe 
the entire watering hole. 

Figure 7B shows the control and video paths of the Figure 7 A system 
combined with system shown in Figures 3A and 3B. The video from cameras 160, 
162, 164, 166 is communicated to computer 170, in compressed or uncompressed 
form on path 122. The computer 170 communicates the video to communications 
network 120 for reception by the web site 140 (Figures 3A, 3B). Preferably the video 
is digitized and compressed by either the cameras 160, 162, 164, 166, the computer 
170, or an external analog-to-digital converter (not shown) and compressor 136 
(Figures 3A, 3B) prior to transfer to the communications network 120. 

Camera control commands are received by the computer 170 on control line 
126, as shown in Figures 3A, 3B and 7B. The commands are formatted, if necessary, 
by computer 170 and transferred to control units 105 attached to cameras 160, 162, 
164, 166. The control units 105 are connected to spin, zoom, or otherwise control the 
cameras as directed by the user. 

Communications links 124 and 122 may be wired, wireless, digital or analog, 
and computer 170 may be located nearby or remote from the site 102. 

The system of Figures 7A and 7B are unlike the embodiments shown in 
Figures 4-6, because each user is assigned a remote camera in the Figure 7A, 7B 
embodiment. Since each user must be assigned their own controllable camera, users 
will have to contend for available cameras. The number of controllable cameras may 
range from a single camera to any number, and is preferably statistically determined 
to correlate to the average number of users who access the web server 140 at any 
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given time or at peak times. The number of cameras may be reduced by using 
known systems which utilize queuing, reservations, and time limits. 



4. Perceived Camera Control Using A Single Camera And A Wide- Angle 
Lens. 

Figures 8A and 8B show another embodiment, using only a single camera, 
where an unlimited number of users may view any portion of the remote site 102. 
This embodiment uses a spherical lens 182 in optical communication with the 
camera 180. The remote site 102 shown in Figure 8 is a remote watering hole or oasis 
as in Figures 7 A and 7B. 

As shown in Figure 8A, a camera 180 has a spherical (or other wide angle) 
lens 182, which provides a 180° spherical (or other wide-angle) view. This view, 
which is communicated to a computer 184, contains distortion. The computer 184 
communicates and compresses the distorted video back to the web site 130 or 140 
which stores and may process the image. Rather than using the computer 184, a 
simple transmitter may be used to convey the entire spherical video to the web site 
130, 140 (Figures 2 and 3). By using appropriate image processing software, the web 
site removes the barrel distortion and stores data relating to the entire spherical 
view. Users may then access different portions of the 180° sphere. In this 
embodiment, the toolbar 151 of Figure 5 is also used. By using the toolbar 151, users 
may move across the spherical view and obtain the "perception" of camera control. 
This embodiment is advantageous in that it can provide the perception of camera 
control to any number of users simultaneously using only one remote camera. 
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Figure 8B shows alternative embodiments of the system shown in Figure 8A. 
As shown in Figure 8B, the spherical (or other wide angle) lens 182 is used with 
video camera 180", which conveys video information to computer 184. Computer 
184 communicates the video over communications network 120 to the web site 130. 
The web site 130 may store or process the received video, and make the video 
available to users at user terminals 302, 304, 306, 308, 310 by communicating the 
video over communication network 125. Communication network 125 is explained 
in more depth below with respect to Figure 10. 

Because wide angle lenses generate distortion, processing is conducted on the 
distorted image to remove the distortion from a segment of the image. This 
processing may be performed at the computer 184, or the web site 130, but is 
preferably performed at the user terminals 302, 304, 306, 308, 310. 

Thus, the web site 130 has available wide angle video for sending to users. 
Users display and view only a segment of the wide angle video at a time. Then, by 
using toolbar 151 (Figure 5), the user may select adjacent segments of the video for 
view. When a user selects an adjacent segment of the video for display, the adjacent 
segment is processed to remove distortion and then displayed. Displaying the 
adjacent segment gives the appearance that the camera was physically "moved" to 
the adjacent side of the original segment. 

One system for electronically removing the distortion from a segment of an 
image obtained from a fish-eye lens is disclosed in U.S. Patent No. 5,185,667, issued 
February 9, 1993 to Zimmerman, incorporated herein by reference. Zimmerman's 
apparatus uses the following hardware for processing a captured and digitized 
image: a microcomputer connected to a remote control, computer control, X-Map 

22 



WO 99/12349 PCT/US98/18271 

and Y-Map; an input image buffer connected to the X-Map and Y-Map with an 
output connected to an image filter and an output image buffer. This hardware, for 
example, or any other suitable hardware, may be placed at the computer 184, or the 
web site 130, but is preferably located at the user terminals 302, 304, 306, 308, 310. 

As a preferred alternative, the specialized hardware is removed and the 
hardware functionality is implemented in software at the computer 184 or web site 
130, but preferably the software is loaded into the user terminal 302, 304, 306, 308, 
310. Thus, in accordance with the present invention a spherical (or other wide- 
angle) image is supplied to the user's terminal, which executes appropriate software 
(which may be a "plug-in" for a browser application program) for displaying a 
segment of the image (or video) without distortion. Additionally, the distorted 
spherical image (or video) may be saved to a storage medium, either at the user's 
terminal or at the web site, for future loading and viewing* 

Figure 8B also shows how to remove the lens distortion without special 
processing. As shown in Figure 8B, a spherical (or other wide angle) lens 182 is in 
optical communication with a video camera 180'. However, a nonlinear imaging 
sensor 186 is placed between the spherical lens 182 and the video camera 180'. The 
imaging sensor is designed to provide a distorted output which cancels out the 
distortion of the spherical lens 182, and thus an undistorted wide-angle image is 
provided to video camera 180'. Alternatively, imaging sensor 186 may itself provide 
a digital output, making it unnecessary to use a camera 180\ In this case, the 
imaging sensor 186 would be directly connected to computer 184. 

Examples of imaging sensors 186 are disclosed in U.S. Patent No. 5,489,940, 
issued on February 6, 1996 to Richardson et al, and in PCT publication WO 

23 



WO 99/12349 PCT/US98/18271 

96/12862, published June 13, 1996 to Richardson et al., each incorporated herein by 
reference. Other suitable imaging sensors may be used with the present invention. 

The image obtained by the imaging sensor 186 may be undistorted and not 
require further processing. A segment of the image may then be selected for display 
by simply passing the image data to a display device. If the imaging sensor is 
imperfect, further processing may occur to correct for defects in the sensor. 
Additionally, further processing for "zoom" and "unzoom" functions may occur. 
This further processing may take place at the web site 130 or at the user's terminal 
302, 304,306, 308, 310. 

The embodiments of Figures 5 through 8 may be used in conjunction with 
either live audio and video or prerecorded video data (with audio) (shown in 
Figures 1-3). For example, if nothing interesting is happening at the watering hole, a 
connected user may access a stored audio and video clip of a lion attack which 
occurred the day before. If "perceived" camera control is utilized, the stored audio 
and video preferably includes all camera angles (or a wide-angle view), such that the 
ability to pan and zoom is preserved. 

5. Web Site Configuration. 

Figures 9 A and 9B show a more detailed view of the web site, listed as web 
site 140 (Figure 3), but which may also correspond to web sites 112 (Figure 1) and 
130 (Figure 2). The web site 140 is connected to a data communication network 120, 
the internet 242, and direct connections 244. The web site contains transmission 
equipment 210, receive equipment 220, 220/ two compression units 108, 114, a web 
server 200, a router 230, and communication equipment 240. The web server 200 

24 



WO 99/12349 PCT/US98/18271 

itself contains a digital matrix switch 250, a plurality of digital video servers 252, 
252', 252", 252/" a firewall access control unit 254, a database server 256, an audio 
and video storage unit 258, a data storage unit 260, an administrative unit 262, a 
digital matrix switch 264, a camera control unit 268 and a digital video matrix switch 
270. 

The web site 140 is connected to the data communication network 120 by 
transmission equipment 210 and receive equipment 220. As shown, multiple 
receivers 220, 220' may be used. Also, as shown, the receivers may have more than 
one video output. Audio and video signals may also be input to the web server 200 
by videocassette (or other suitable recorded media) or simply by feeding in 
television programming. As with Figures 1 and 3, these signals are preferably 
compressed by compression units 108, 114. On the opposite side, the web server 200 
is connected to remote users by a router 230 and communication equipment 240, 
which in turn are connected to the internet 242 or directly connected 244 to users. 
The communications equipment 240 outputs the video streams 116 through a 
number of input/output ports. 

As previously stated, the web server 200 contains a digital matrix switch 250, 
a plurality of digital video servers 252, 252', 252", 252,"' a firewall access control unit 
254, a database server 256, an audio and video storage unit 258, a data storage unit 
260, an administrative unit 262, a digital matrix switch 264, a camera control unit 268 
and a video matrix switch 270. 

The digital matrix switch 250 receives all incoming compressed video signals 
from the receivers 220, 220' and the compressor units 108, 114. The matrix switch 
250 also receives compressed video data from database server 256. Under control of 
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the administrative unit 262, the digital matrix switch 250 outputs the input 
compressed video signals to digital video servers 252, 252', 252", 252'". In this 
manner, any input signal can be transferred to any video server as directed by the 
admin unit. Also, stored programming from the database server 256 is routed to the 
digital matrix switch 250 to be switched as if it were incoming live video. The 
outputs of the digital matrix switch 250 also connect to the database server 256, so 
that anything at the inputs, such as incoming live audio and video, can be stored in 
the database server 256. 

The compressed input video is passed into various digital video servers 252, 
252', 252", 252"' for formatting. Users who connect to web server 200 preferably run 
their own decompression software so that the no decompression need occur at the 
web server 200. As an alternative, the digital video servers may decompress the 
input video. 

The audio and video from the video servers 252 are passed through a second 
digital (video) matrix switch 270. Since switching has already occurred at the digital 
matrix switch 250, the second video matrix switch 270 is not required, but is desired 
for maximum flexibility. It is also optimal where the number of users exceeds the 
number of video inputs, as one input may be channeled to numerous connected 
users. 

In a preferred embodiment, the matrix switch 270 may contain a processor 
which joins different frames of video and audio such that each output contains 
frames for multiple video pictures (including audio). This enables users to receive 
split screen images of video and select an audio track for playback (see Figure 14, 
discussed below). The split-screen images may be formed by using known methods, 
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which may differ depending on the type of compression used. For example, digital 
images may be decompressed, combined with other decompressed images, and then 
re-compressed; or the images may be decompressed and converted to analog, 
combined, and then converted to digital and compressed for transmission. 

The signals switched by the video matrix switch 270 are preferably digital. 
This is because the communicated video streams 116 are preferably digital. It is 
preferred to process all the signals in the web server in the digital domain to 
improve simplicity and maintain maximum flexibility. 

The various streams of video output from the video matrix switch 270 are 
passed to the firewall access control unit 254 for output to the router 230 and the 
communication equipment 240. 

Using this system, any user may receive any signal present at any input, 
including stored signals within audio and video database 258 or data storage unit 
260. Additionally, any compressed digital signal present at the input to digital 
matrix switch 250 may be stored in the audio and video storage unit 258 or data 
storage unit 260. This is advantageous in the perceived camera control embodiment 
(Figures 4-8) where the web server 200 must output a different video picture to the 
user upon user request. When the user request is received by the web server 200, the 
administrative unit 262 directs the matrix switches 250 and 270 to output the correct 
video stream to the user. If the user is requesting stored video, the administrative 
unit directs the database server 256 to provide the video to digital matrix switch 250. 
If graphics or textual data are required, the administrative unit 262 directs the 
database server 256 to output the text or graphics to digital matrix switch 264. 
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Although shown as one functional box, the database server 256 may be 
implemented by using several servers and/or multiport servers. The audio and 
video storage unit 258 and data storage unit 260 may be implemented by using 
many storage media of different types, such as optical storage devices (i.e. CD- 
ROM), magnetic disks, magnetic tape, or memory circuits (i.e. RAM/ROM). The 
number of units depends on the amount of stored data, the number of users, and the 
desired output speed. The database server 256 may be one or multiple units. The 
audio and video storage unit 258 stores (preferably compressed) audio and video 
presentations, including all relevant camera angles. The video servers 252 may also 
be implemented as one or more servers and /or multiport servers. 

The data storage unit 260 is used to store information relating to audiovisual 
displays. This information relates to the menu structure and screen displays 
communicated to connected users. The stored information may also relate to 
specifically to the audio and video which is currently being displayed and heard. 
For example, in the demolition embodiment of Figure 5, a user may click on a "more 
info" icon, to obtain information on demolition. Such information, which could 
include statistics on dynamite, for example, would be stored as text or graphics in 
data storage unit 260. The "more info" command would be transmitted to the 
communications equipment 240, pass through the router 230, and the firewall access 
control 254 to administrative unit 262. The administrative unit 262 then directs the 
database server 256 to recall the relevant information, such as statistics on dynamite, 
from data storage device 260 and pass the information to digital matrix switch 264. 
The recalled information is then passed to the firewall access control unit 254, the 
router 230, and the communication equipment 240 for transmission to the proper 
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subscriber. The data may be combined with audio and video in the firewall access 
control unit 254, or be a separate transmission. 

In the perceived camera control embodiment, the communication equipment 
240 forwards the user's command (such as "pan right") to the router 230, which 
detects the command and forwards it to the firewall access control unit 254, which 
passes it to the administrative unit 262. The administrative unit 262 controls the 
video being fed to each connected user. The administrative unit 262 also responds to 
user commands by instructing either the matrix switch 250 or the matrix switch 270 
to pass a different audiovisual signal from another source (i.e. camera, for example, 
the camera to the right of the present camera) to the connected user. If the user is 
receiving a stored image from database 258, the administrative unit instructs the 
database server 256 to recall the appropriate video signal. 

In the actual camera control embodiment (shown in Figures 3 and 7), 
commands from the user (such as "pan right") are received by the communication 
equipment 240 and forwarded to the router 230. The commands enter the web 
server 200 via the firewall access control unit 254, and are passed to the 
administrative unit 262. The commands may be stored in the administrative unit 262 
or passed to the database server 256. Either way, the commands pass through the 
camera control unit 268 which f ormats^e commands as necessary for remote 
camera control. The formatted commands are passed to the transmission unit 210. 
The transmission unit 210 provides the commands to data communication network 
120 for reception at remote cameras and CPU 134 (Figure 3). 

In the spherical (or other wide angle) lens embodiment (shown in Figures 8A 
and 8B), where the remote camera uses a spherical lens 182, the administrative unit 
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262 determines which segment or quadrant of the audiovisual image is to be 
supplied to the user in response to the user's command. In this embodiment, the 
spherical image is stored in database 258 prior to being output to digital matrix 
switch 250. The image is split into a number of sections, which when combined form 
the entire 180° sphere. By using suitable image processing software, the distortion is 
removed or minimized in each segment. The administrative unit 262, in response to 
a user command, determines which segment of the sphere should be sent to the user. 
The administrative unit then directs the database server 256 to retrieve and output 
the correct segment to the digital matrix switch 250. By controlling the digital matrix 
switch 250 and video matrix switch 270, the administrative unit 262 is able to ensure 
that the user receives the correct segment of the spherical image. 

However, as previously stated, in one preferred embodiment the entire 
spherical (or other wide angle) video is communicated to the user, and the distortion 
removed by software at the user's terminal. This minimizes the complexity of the 
processing necessary at the web site 140, and allows the user to store the entire 
spherical (or other wide angle) video. 

Preferably, the communication equipment 240 is designed to automatically 
determine the maximum data rate at which information can be transmitted to the 
connected users. The data rate depends on the type of connection the web sites has 
with the user, and the type of equipment the user is operating. In one embodiment, 
the communications equipment uses the maximum data rate possible as sensed from 
the user's communications. Alternatively, users may enter their data rates when 
prompted by a menu screen, as shown in Figure 15 and described below. The data 
rates are then stored in communications equipment 240. The communications 
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equipment 240 may also compress the video streams prior to transmission using any 
known compression algorithm. Additionally, the communications equipment may 
remove video frames, preferably prior to compression, such that the resulting data 
rate is reduced to be compatible with the user. 

Figure 9B is identical to Figure 9A, but contains an input interface 225 and an 
output interface 235. The input interface 225 is used to obtain digital video from 
other sources, such as a paging system, cellular system, cable television system, etc... 
The output interface connects the web site to other communications systems such as 
paging systems, cellular systems, or cable television systems. In the case where the 
input interface connects to an analog system, it contains suitable analog to digital 
converters (not shown). Also, where the output interface connects to an analog 
system, it contains suitable digital to analog converters (not shown). 

For example, the input interface 225 may obtain images or video from a 
paging system, and the output interface 225 may be connected to a paging system to 
broadcast video or images to a selective call receiver. In this regard, the following 
publications are incorporated by reference, each of which relates video/images to 
selective call receivers: PCT Publication No. WO 96/07269, published March 7, 1996 
by Jambhekar et al., PCT Publication No. WO 96/21173, published July 11, 1996 by 
Harris et al., and PCT Publication No. WO 96/21205, published July 11, 1996 by 
Harris et al. 

6. Communication to the User Terminals. 

Figure 10 shows how the users are connected to the web site, and shows an 
example of a communications network 125 (Figure 8B) in detail. The connections 
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shown in Figure 10 apply to the web sites of the previous figures, including the web 
site 112 (Figure 1), 130 (Figure 2) and 140 (Figures 3 and 9). Figure 10 shows a server 
platform 200, the internet 242, two direct connection 244, two traditional internet 
hosts 272, 274, two cable internet hosts 276, 278, a satellite-based internet host 280, a 
telephone dialup 282, an ISDN channel 284, a cable plant 286, 288, a satellite system 
290 and a plurality of connected user terminals 302, 304, 306, 308, 310. 

In operation, the web site 112, 130, 140 may communicate over the internet 
242 to a number of different systems. These systems include a traditional internet 
host 272, 274 and a cable headend internet host 276. The traditional internet host 
272, 274 may be connected via a telephone line 282 or an ISDN channel 284 to a 
plurality of remote user terminals 302, 304, respectively. The cable internet host 276 
may be connected via a cable plant 286 to a remote user 306. 

Alternatively, the web site is connected via a direct connection 244 to a cable 
headend internet host 278 or satellite-based internet host 280. The cable headend 
internet host 278 communicates to a cable plant 288 and a remote user terminal 308. 
The satellite-based internet host 280 communicates via a satellite 290 to a user 
terminal 310. These direct connections 244 enable a higher data rate and use a high 
speed cable modem. 

It is advantageous that the communications equipment 240 (Figure 9) enables 
communications with any type of user terminal no matter what the data rate or 
system. Of course, user terminals with higher data rates will receive higher quality 
audio and video images. 
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7. Exemplary Screen Displays and Features. 

Figures 11-16 show examples of display pages which are shown at the remote 
user's terminal. The pages and menus are stores in data storage unit 260 (Figure 9) 
as graphical and/or textual information. 

Figure 11 shows an example of a home page, using advantages of the present 
invention. The home page 400 contains a number of advertisements 402, numerous 
web links 404, a society link 406, options for viewing television programming 408, a 
plurality of rapid access entry options 409 including a "World Watch Live" option 
410, and options for clubs 412. 

The advertisements 402 are useful for the page provider to generate revenue. 
As described previously, the system is designed such that television programming 
can be supplied over the internet. Users may view television programming by 
selecting the home page television option 408. The Magazines 404 are used to 
provide information concerning specific topics to the user. Users may join a society, 
having additional membership benefits, through the "society" selection 406. The 
"World Watch Live" feature 410, part of the rapid access entry options 409, is 
selected when users wish to watch live video from remote sites. The clubs shown in 
the club option 412 are selected by users who wish to obtain information related to 
common areas of interest. 

Figure 12 shows a society menu 406, selected from the Figure 11 home menu 
page. As shown in Figure 12 there are options for "World Watch Live" 420, there is 
an advertisement 402, subscription information 424, and numerous club options 422. 
This screen and all the functions selected in response to the displayed options may 
be provided on a subscription or temporarily free basis. 
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Figure 13 shows one example of a "World Watch Live" menu 440. This menu 
is used to select remote locations from which to observe live or prerecorded video. 
In this example, a map of the world is presented with sites that are available to select 
for observing live video. The screen indicates sites that are active 442 or under 
construction 444. This menu also contains two advertisements 402. 

The "World Watch Live" embodiment allows connected users to visit virtually 
anyplace in the world to learn more about its culture, geography, or environment. 
Coupled with perceived or actual camera control and associated prestored video, 
textual and graphical information, a powerful and inexpensive learning tool is 
realized. This is more closely shown in Figure 14. 

Figure 14 shows a menu 450 which corresponds to the Egyptian site in Figure 
13. This screen concerns "Giza, Egypt", and contains live video from five cameras. 
As shown in the screen, mere is camera one 452, cameras two through five 454, a 
"Map" option 456, an "About This Site" option 458, an "About Egypt" option 460, an 
"Upcoming Events" option 462 and a "Remote Control" option 464. Camera one 452 
is the default for the main viewing camera. The user may select video image sizes 
and the number of images to be displayed, limited by the equipment the user is 
operating. Video from cameras two through five are supplied along with that from 
camera one to provide alternative sites and viewpoints about the topic of the screen 
(i.e. Egypt). 

The "Map" option 456 brings the user back to the world map (Figure 13) to 
select additional sites. The "About This Site" option 458 brings up text, graphics or 
additional video concerning the site of Giza, Egypt. For example, a professor 
appears and talks about the origin of the Sphinx (shown by camera 1). The 
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embodiment shown in Figure 16 and described below (interactive lecture) may be 
combined with the "About This Site" option. Additionally, other video may be 
displayed in response to selection of "About This Site". Such video may be a 
documentary of the Sphinx or discussion about the technology that historians 
estimate was used to construct the Sphinx. 

The "About Egypt" option 460 brings up graphics, text or additional video 
concerning Egypt. For example, a map of Egypt with population densities may be 
shown. The option for "Upcoming Events" 462 brings graphics, text or video 
concerning new events in Egypt. For example, text and newspaper articles 
concerning the construction of new irrigation canals is displayed. 

"Remote Control" option 464 brings up a command menu (such as the "tool 
bar" 151 of Figures 5A-D) which allows the user to change camera angles or 
positioning in any of the cameras capable of that effect. The menu would apply to 
actual or perceived camera control. For example, the user could pan around the 
Sphinx (camera 1, shown at 452) to observe it from the front, each side, and back. 

Thus, this single screen relating to Egypt provides a wealth of information at 
a single internet address (or web site). It is unnecessary for a user to "link" to other 
locations on the internet. Audiovisual presentations are displayed, which give the 
user insight into the people and culture of Egypt. Text, graphics, and additional 
stored video is available to further educate the user. Camera control (actual or 
perceived) gives the user the feeling of walking around different locations in Egypt. 

Figure 15 shows a screen 470 which asks users about their equipment in order 
to determine the appropriate data rate for communications. Preferably the screen is 
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not needed and the data rate is determined by communication equipment 240 
automatically. Note that an advertisement 402 is also shown on this screen. 

Figure 16 shows an interactive lecture embodiment of the present invention. 
As shown in Figure 16, live video 500 of an astronomy professor's lecture is 
transmitted to connected users. The users are able to ask the professor questions 510 
and receive answers 512. The live video 500, questions 510, and answers 512 are 
shown to all connected users. Preferably, the users enter questions via keyboard or 
microphone. However, if suitable data rates are available, the user may ask a 
question via video. Thus a split screen video showing both the person asking the 
question and the lecturer may be presented to all users simultaneously. The answers 
are preferably given by the lecturer, who may observe the question on a remote 
display. Alternatively, the answers may be supplied by the web site as text, 
graphics, or prestored video. The answer may pass through a closed captioning 
device, be encoded, and displayed on the screen in an answer box 512. 

Referring to Figure 9A, questions are sent to the web site 140 as part of the 
normal user terminal communication. The web site 140 receives the question at the 
communications equipment 240 and forwards the question through router 230 and 
the firewall/ access control unit 254 to the administrative unit 262. The 
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